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Human Cyber-Physical Systems (h-CPS) 


CPS that operate in concert with humans 



Driver Assistance in Cars 



Robotic Surgery <& 
Medicine 



Fly-by-wire Cockpit 
Interfaces 



UAVs with Human 
Operators 



Semi-Autonomous Manufacturing 
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Overall Project Objective of VeHICaL 


To develop a science of verified co-design 
of controllers for semi-autonomous 
cyber-physical systems and 
interfaces between humans and 
cyber-physical components 
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Motivating Applications 


• Semi-Autonomous Automobiles 

TECHNOLOGY Sljc iNcUi JJork eiincs 



The 15-Point Federal Checklist for Self-Driving Cars 


A Lesson of Tesla Crashes? Computer Vision 
Can’t Do It All Yet 


• Semi-Autonomous UAVs 


FAA Expects 600,000 Commercial 
Drones In The Air Within A Year 
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HOW TO ACE THE FAA’S NEW TEST AND 
BECOME A PRO DRONE PILOT 
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Project Goal: Develop Science of Design of h-CPS 
with Provable Guarantees on Behavior 

Some Major Unsolved Problems to be Tackled: 

• Effective Sharing of Control between Human and 
Autonomous Controller 

• Modeling Human Cognition, Perception, and Action, 
and developing Sensing Strategies to infer and 
evaluate such models 

• Uncertainty in h-CPS Dynamics and in Modeling: need 
new techniques blending Verification, Learning, and 
Control 

• Verifiable Human-Computer Interface Design 
integrated with Control System Design 

• Privacy-preserving Learning of Models of h-CPS 
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Sensing & HMI Design 


Formal Modeling & 
Specification 


Learning Models 
from Data 
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Security & 
Privacy 



Verification 


Control 
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System-level 
Integration & 
Validation 


Evaluation & 
User Studies 
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Four Project Thrusts 


1. Specification and Modeling 

2. Learning, Verification, and Control 

3. Human-Machine Interface Design & Verification 

4. Testbed and Evaluation 
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Specification for h-CPS: What does "correctness 

mean? [Li, Sadigh, Sastry, Seshia, TACAS'14] 


• Safe & Correct Autonomy 



- Autonomous mode should be proved correct 


• Effective Monitoring 

- Must monitor all relevant environment variables 


• Prescient Switching 

- Sufficient lead time to switch between human & auto 


• Minimally Intervening 

- Should rarely request human intervention 

• Other Properties: 

- Security & Privacy 

- HMI Properties: Absence of Mode Confusion, ... 
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Correct-by-Construction 
(Semi-Autonomous) Control 



Goal 

Environment Model 
Human Model 



Controller 

Synthesis 


Realizable 


Initial work for: 

• Linear temporal 

logic (LTL) [Li, 
Sadigh, et al/14] 


Repairabli 





Unrealizable 


Diagnosis 


"N 


• Signal temporal 
logic (STL) 

[Raman, et al/15; 
Ghosh, Sadigh et 
a 1/16] 


Repair 

& 

Intervention 




Unrepairable 


Ask for human 
intervention 
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Human-Aware Control of Vehicles 

[Sadigh, Sastry, Seshia, Dragan; RSS, IROS 16] 




Leverage human 
responses to estimate 
human internal state. 


SAMPLE RESULT 



bt + 1 (0)ocb t (0)-p(u H \x t ,9, 

Info Gatherii 

As 


r R (x t ,u H ,d,u R ) = K(b t )-K(b t 

^ ^ goal (X t , U-h> ■ > Uji) 

y V ' 

Goal 

u R = argmaxE d [R R ] 

Ur 



Lane Change Nudging In 



Distracted 
luman 


Attentive 

Human 


S. A. Seshia 


12 






Human Modeling: Data-Driven Modeling 
and the "Bounded Optimal" Model 

• Uncertainty in human behavior and variation (person 
to person) are central aspects of h-CPS 

- Probabilistic formalisms, learned from data 

- Consistent with formalisms using in verification and control 

• Model human behavior based on "Resource 
Rationality" [Griffiths, Lieder, & Goodman, 2015] 

- Trade off error with the cost of computation, assuming an 
abstract computational architecture 

- an instance of "bounded optimality": choosing the best 
algorithm to solve a problem (Russell, 1995) 
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Resource rationality in VeHICal: Directions 



• Prediction: Modeling the allocation of cognitive 
resources in dynamic environments 

- e.g., do we make rational choices about when to check our 
texts when driving? 

• Inference: Developing models that can be used to 
infer when/how humans are resource-limited 

- e.g., behavior patterns that result from having less 
resources for tracking, relevant to prescient switching 
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Experimentation on CPS Testbeds 

H 

• Driving Simulator Testbed: Force Dynamics 
401CR (industrial simulator with motion feedback) 

• Automotive: Partnerships with automotive 
companies 

• UAVs: Tomlin lab, partnerships with drone 
companies 

• Jacobs Hall: State-of-the-art labs for rapid 
prototyping of interfaces and hardware 




Initial work [Rezvani et al., ITSC' 16 ]: 

Impact of user interface 
design on transfer of control 
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Education and Outreach activities 


• Joint development and dissemination 
of educational materials on h-CPS 



Cyber-Physical Systems 


- Online course on h-CPS jointly offered 
by Pis 

• Joint mentoring of postdoctoral 
researchers 



• Summer programs for undergraduate 

and high/middle school students 

- Projects conceived of jointly and offered 
in a coordinated manner to amplify the 
overall impact 
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and Learning 

Scientific Impact: 

• Developing a Science 
of Co-Design of 
Human Interfaces 
and Control 

• Bridging Model- 
Based and Data- 
Driven Design of CPS 


Broader Impact: 

• Significantly improve 
safety, security, and 
performance of 
systems where humans 
interact closely with 
automation 

• Involve middle/high- 
school and 
undergraduate 
students in VeHICaL 
activities 
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VeHICaL: Verified Human Interfaces, Control, 

for Semi-Autonomous Systems 


Challenge: 

• Co-design human 
interfaces and control for 
human-cyber-physical 
systems with provable 

guarantees 

• Apply to semi- 
autonomous vehicles 
(ground and air) 

Solution: 

• Integrate Learning, 
Verification and Control 

• Bounded Resource 
Rational Human 
Modeling 

• Prototype Controllers & 
Interfaces, Evaluate on 
Testbed 





